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Network-Based Service to Provide Oo-Demand Video Summaries of 

Television Programs 

Field of the Invention: 

s This invention relates to extracting summaries of television (or audio) 

programming both during live broadcasting and post broadcasting. It is in one instance 
concerned with recipient controlled recovery of summaries of programming supplied via 
cable and xDSL access links. In another aspect it concmis back-and-forth intermingling 
of summaries and live/recorded programming under control of a recipient customer. It 

10 particularly concerns apparatus within a broadcast networic and on customer premises to 
enable such extraction and presentation of summaries. 



Background of the invention: 

Recording apparatus presently goierally available permits a recipient of television 
IS and radio programming to record real-time programming (i.e., on VHS tapes) and replay 
it at a later time. The most ubiquitous example is the VCR, which allows recordation of 
live TV programming under recipient control for replay at a time of the recipient's 
choice. To record, the customer must select the time of the interval to be recorded. 
There exist more sophisticated systems (Digital Video Recorders) which 
20 automatically record programming (via a digital cache/memory) and allow the user to 
selectively view portions of the recorded programming at a later time. An example of 
such a recording system is the ReplayTV digital video recorder and the TiVo Personal 
TV digital video recorder, which supply up to 30 hours of recording time. They provide 
features that surpass recording by monitoring listener preferences and by suggesting 
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programming appropriate to these preferences. Recording may occur during live 
programming and if live viewing is interrupted the viewer may recover a delayed 
presentation of the programming at the end of the interruption interval. 

These record/playback systems are however limited by failing to fully utilize 
5 network system capabilities* Summaries, for example, may be poorly defined or 
delimited resulting in loss of program material available to the user/viewer. Options 
available to the user are limited to manipulation of programming as received rather than 
using network and program provider resources to provide a broad spectrum of control. 



10 Summary of the Invention: 

Recipients who receive network programming are enabled, according to 
principles of the invention, to receive television/audio programming &om a distribution 
network and view selected summaries of such programming as part of a network-based 
service. Identification and preparation of the summaries is at least in part a network- 

15 controlled process with recipient utilization of the features provided being under control 
of the recipient viewer. Features include recovery of audio, snapshots (stills) and full- 
motion video summaries extracted from program content. A summary in some instances 
may be a specially^created overview, of a program created by a program source that is 
provided by a program source or provider. In some instances, segments may be created 

20 independent of the programming as a leader or overview in advance of the program or as 
an inducement to watch pre*stored programs. 

Smnmaries are generally program segments extracted from complete programs 
and may include various combinations of audio, full-motion video, still pictures (i.e., 
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selected frames), and otha presentation modes either singly or in any desired 
arrangement Accessing techniques may be by user control or system controlled 
selection based on user profiles. Direct user selections, may be made by a user for 
covering interruptions to user viewing. Network control allows greater selectively, to 

S determine if program content is desired or to meet other user generated requests, which a 
summary may satisfy. 

In an illustrative embodiment, the sources of programming generate and transmit 
summaries that include control information to permit a recipient to selectively choose and 
replay the simmiaries. In some embodiments a video summary server (VSS) is included 

10 in the network to perfonn suinmary/program storage and provide access fimctio In 
another embodiment a set top box (STB) is included on recipient premises to receive 
program/sununaries and perfonn some of the viewing control functions and store 
summaries. 

The process of transmitting and generating summaries is based in part upon 
IS properties specified in the MPEG-2 (Moving Picture Experts Group) standards. MPEG-2 
is a standard concerning signal encoding and real time transmission of video and audio 
program streams. The standards primary goal is bandwidth compression as well as 
combining and multiplexing. Packet headers under the standard include many features 
such as time stamps, typing of payloads and combining/multiplexing all of, which are 
20 usefiil in enabling the invention of applicants. MPEG-2 includes many features including 
information-carrying capability of a video stream. This permits the transmission of 
information by the program source that enables the features of the invention to be 
realized. 
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In one aspect the method of the invention processes programming to facilitate 
selection and delivery of summaries of the programming to recipients by providing the 
programming to the recipient via a program channel. Index markers are applied to the 
program to divide the programming into segments. Summary segments are generated 

S containing information from corresponding programming segments and related to one 
another by corresponding index markers. Metadata files are created and associated with a 
simmiary channel and operate to delimit beginning and ending of segments in both 
programming and summary channels. Metadata includes the indexing information for 
facilitating links between programming segments and summary segments. A 

10 user/recipient selects a summary by activating a link between a programming segment 
and a summary segment by utilizing the metadata file included with the summary . 
channel. The summary is transmitted to the recipient, via the summary chaxmel. 

In a particular embodiment the summary channels include metadata to 
implement/facilitate a linking back and forth between summary and program channels 

1 s under command of a user. The MPEG-2 standards enable user data, in the form of 

metadata, to be included in a video sequence. This data delimits the beginning and end of 
segments in both summary and program channels. The standard fiirther allows an 
elementary data stream to be included within a program stream. The elementary data 
stream is used to provide indexing used to implement video linking and interrupted 

20 viewing features. 

In one aspect the relevant control is responsive to metadata supplied to define and 
identify summaries corresponding to particular programs. It defines start and end marks 
of both summaries and programs and the nature of the linking connections between 
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summary and program. Such metadata is easily to integrate with the programming under 
the MPEG-2 standard. 

The reviewing and selection functions include but are not limited to; 



Supplying and viewing summaries of programs as a means of selection of 
S programs to view in their entirety. 

Providing recipient control to select related sununaries and full program segments 
through summary-segment linkage marks. 

Providing one-way and two-way video hyperlinks between programs and 
summaries. 

10 Providing the viewing recipient with missing portions of interrupted program 



« • 



viewing. 



Generating summaries during live programming for intennittoit viewing. 
Providing summaries permitting selective recipient recording of portions of a 



prog] 



Description of the Drawing: 

FIGs. 1 through 6 are diagrammatic illustrations of modes of selections and 
presentations of sumrnaries of video programming; 

FIGs. 7 and 8 are schematics of a screen control menu presented to a viewer; 
20 FIGs. 9 and 10 are schematics of video summary system components involving 

cable networks; 

FIG. 1 1 is a schematic functionally illustrating a set top box (STB) with 
enhancements for enabling siunmary storage and retrieval; 
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FIG. 12 is a schematic of video sununaiy system components added to a network 
having a xDSL access network; 

FIG. 13 is a process diagram describing summarization of live television 
programming; 

S FIG. 14 is a flow chart describing a monitoring/control of video storage 

processes; and 

FIGs. IS, 16 and 17 is a flow chart disclosing a control process permitting the 
exercise of summaiy selection and viewing by a recipient of programming. 



10 Detailed Description: 

Viewing of simimaries of audio and video programming is available, according to 
the invention in many styles and formats. Summaries may be generated dynamically by a 
recipient viewer's requests or recalled firom a central repository of summaries at user 
request Various systems of generating, using and recalling summaries are illustrated in 

IS the program schematics of FIGs. 1 through 6. In the circumstances of FIG. 1 a program 
sequence is shown by the bar 101 showing the evolving of the program. As the program 
progresses the program's progress is marked by the maricers PI, P2, P3, . . . These 
markers are related to corresponding markers shown in the Summary (audio) bar 102 and 
the Summary (video) bar 103. As shown the program from 0 to PI may be shown as 

20 summary from 0 to SI. The subsequent portion from PI to P2 may be bypassed and a 
subsequent portion of the program from P2 to P3 shown as summaiy from S 1 to S2. The 
summaries shown are full motion and all inclusive of materials within the marker interval 
covered and/or summarized. 
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In the illustrative summaries of FIG. 2» markers PI » P2, P3^ . . . segment the 
program. An audio summary 201 is created resulting in audio simmiaries 0 to S3 and S3 
to SS. The video portion is reduced to snapshots or still pictures 203 with each still 
picture related to a particular program segment. The snapshot fix>m position P2 in the 
S program is used during the summary interval from S 1 to S2. The use of still images and 
full motion images may be interspersed as desired. This option is illustrated in FIG. 3 
where the video summaries 303 comprise still images and full motion images of the basic 
programming 301 . Full motion images are interspersed with still images. 

FIG. 4 illustrates an example of one-way video hyperlinks. A user begins by 
10 viewing a simmiary and selecting the link function during the summary segment 405. 
Control passes to the beginning of the corresponding program segment 415. When that 



program segment completes, control continues with the ronaining program segments. 

One-way video hyperlinks can also pass control from a program segment to the 
beginning of the corresponding summary segment When that summary segment 

IS completes, control continues with the remaining summary segments. 

FIG. 5 illustrates the operation of two-way hyperlinks. A user begins by viewing 
the summary and selects the link function during the summary segment 505. Control 
passes to the begiiuiing of the corresponding program segment 515. When that program 
segment completes, control automatically returns to the beginning of the next summary 

20 segment 506. The user again selects the link function during summary segment 507. 
Control passes to the beginning of the corresponding program segment 517. When the 
program segment completes, control automatically returns to the beginning of the next 
summary segment 508. 




Bhagavath; Markowitz; O^eil 
1999-0494 

FIG. 6 also illustrates the operation of two-way hyperlinks. A user begins by 
viewing the program and selects the link function during the program segment 615. 
Control passes to the beginning of the corresponding summary segment 60S. When that 
summary segment completes, control automatically returns to the beginning of the next 
5 program segment 616. The user again selects the link function during program segment 

617. Control passes to the beginning of the corresponding summary segment 607. When 
that summary segment completes, control automatically returns the beginning of the next 
program segment 618. 

The relationship between program and summary segments may be established and 

10 controlled through use of a metadata file or files. In the illustrative embodiment, 

metadata is used to identify program and summary segments. It also establishes the flags 
useful for two-way and one-way connecting links for use in changing firom a program to 
a summaiy or vice versa. A one way link allows a user to switch fiom a summary 
segment to a corresponding program segment or switch firom a program segment to a 

IS corresponding sununary segment A two-way link allows the user to view a sununary 
segment and switch to a corresponding program segment with an automatic return to the 
next simimary segment when the corresponding program segment completes. Two-way 
links also permit.switching from a program segment to a corresponding summary 
segment with an automatic return to the next program segment. 

20 The metadata file may also provide key words for each sununary segment. Start 

and end positions for each summary segment may be established and correlated with start 
and stop positions for each program segment. Summary metadata may be expressed in 
XML (extensible Mariaip Language) or a suitable equivalent. XNfL is well known and 
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need not be discussed in detail. An illustrative XML sununaiy file may assume the 
following fomi, to select specific subject matter fiom, a program. 

<summaiy> 

<summaryjd>726746425</summaiyjd> 
<programJd>399868815</programJd> 
<links>two-way</links> 
<summary_segment> 
<index>0</index> 

<keywords>Federal Resenre, Interest Rates</keywoids> 
<start>0</start> 
<end>1000</end> 
<summary_segment> 



<pro9am_segment> 
<index>0<;/index> 
<starfX)</start> 
<end>10000<yend> 

<pi9gram_segment> 



<ysummaiy> 
etc. 

The foregoing illustrative metadata file indicates that there is a summary of parts 
of a program relating to the federal reserve and interest rates. The metadata file is 
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generated by the conteat provider. It permits the system to select the desired summary 
information if available. Transmission of the signal in accord with the MPEG*2 standard 
allows the interaction with the programming file by means of this metadata file. 

The sections of this metadata file are now described. First the <summary> tag 

S indicates the start of the file. Second the <summary id> tag provides a globally unique id 
for this sunmiary. Third, the <program id> tag provides a globally unique id for the 
corresponding program. Fourth, the <links> tag indicates that the links between the 
program and summary segments operate as two-way links. Fifth, the <sxunmary 
segment> tag indicates the start of a siumnary segment definition. One metadata file can 

10 define multiple siunmary segments. Sixth, the <program segment> tag indicates the start 
of a program segment definition. One metadata file can define multiple program 
segments. 

Each <summary segment> contains four tags. The <index> tag assigns a unique 
index to that segment. The segments in a summary are assigned indexes that begin at 
15 zero and increase sequentially. The <keywords> tag associates several keywords with 
this summary segment. Software can use these keywords to identify segments that are 
most likely to be of interest to a particular viewer. The <start;> tag indicates the position 
in the summary that marks the begiiming of this segment. Position can be specified as a 
frame number. The <end> tag indicates the position in the summary that marks the end 

20 of this segment. 

Each <program segment> contains three tags. The <index> tag assigns a unique 
index to that segment The segments in a program are assigned indexes that begin at zero 
and increase sequentially. The <start> tag indicates the position in the program that 
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maiks the beguming of this segment. Position can be specified as a frame number. The 
<end> tag indicates the position in the program that marks the end of this segment. 

Each summary is sequence of summary segments. The segments are contiguous 
and non-overiapping. Similarly, each program is a sequence of program segments. The 

5 segments are contiguous and non-overlapping. 

A viewer exercises control, in one embodiment, by interaction with a set top box 
(STB) or with a video summary server (VSS) by means of an interactive menu such as 
illustrated in the screen display of FIG. 7. Here the list of summaries in one example of 
the invention may be supplied over a channel used exclusively (i.e., an S chaimel) for this 

10 purpose. In the illustrative menu the viewer may select summaries related to the listed 
programs by entering the nimiber associated with that summary in the summaiy id box by 
a remote or other means. The menu of FIG. 8 provides for summaries to recover material 
lost due to intermpted viewing of a program. Intenrupted viewing occurs when a viewer 
is interrupted in his viewing (i.e. called away from the television receiver). If a viewer 

1 s wishes to subsequently view a summaiy of missed programming content or view the 
current program, a selection may be entered by entering the appropriate control number 
in the option id. This process may be &ciUtated by use of a dedicated I chaimel. 



Thei 



is accomplished, in part, with an application of enhancements 
to various elements in the program delivery system. In the illustrative embodiments such 
20 enhancements are made to the set top box (STB) in a cable reception system or to a video 
summary server (VSS) in xDSL program delivery systems. These enhancements provide 
interface capabilities and infoimation availability for program selection and the process 
of delivering summaries. 
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An illustrative system architecture for constructing and viewing program 
summaries is shown in the Fig. 9. This system is adapted for Cable networks and as 
shown includes a television-broadcasting source 901 connected to a network POP (Point- 
of-Presence) 903. POP herein is the physical equipment of a provider network at which 
the network terminates and which is connected to a recipient subscriber. A cable access 
link 905 connects POP 903 to an enhanced STB 907 located at customer premises 909. 
STB 907 provides the television signals to the television receiver 91 1. A server 913 
maintains billing records. 

A series of signals, for summary viewing, is diagrammed in the schematic of FIG. 
10, showing the primary signal interfaces between the television-broadcasting source and 
the customer. The television-broadcasting source 901 provides program channels 1001 
via the POP 903 to the STB 907 at the customer premises. The television channels are 
transmitted by various program providers to the POPs 903 and transmitted ftom the POPs 
to the STBs 907 at the customer's premises 909. 

Each program and summary is assigned a globally unique identifier. This 
information is transmitted to the VSSs and STBs by the program summary channel. 

Each broadcaster maintains a schedule of programming, which is accessible to the 
STBs via IP unicasting 1002 through the POPs 903. The customer may access 
programming-related descriptive material via the same IP channel. Separate digital 
suimnary channels 1003 are used to transmit program summaries from the broadcasters to 
the STB. The STB can receive and store these summaries from the broadcaster. 
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Charging data for the service provided is maintained by a billing system server 
913 connected via channel link 1004 through the POP 903 to the STB 907. Billing in the 
illustrative example may be by subscription, by transaction or other suitable arrangement. 

An illustrative enhanced set top box (STB) 907, for cable reception, is 
schematically shown in block form in the FIG. 1 1 . Program and summary channels from 
the broadcaster are applied to a bank of AudioA^ideo RF CATV Demodulators 1 101. 
The demodulators each recover a specific program or summary channel as directed by 
control of the software controller 1 103. Software contioUer 1 103 includes stored 
programming that controls and coordinates operations of the various STB functions. 

The recovered programming/summaries (sti-eams) are applied to an MPEG-2 
Decoder. MPEG-2 is a standard concerned with video and audio coding and processing 
other multimedia signals (e.g., packet streams). Decoder 1105 is designed in accord with 
this standard. MPEG processing, decoding of digital signals and design of a MPEG-2 
system decoder is discussed in "Digital Video: An Introduction to MPEG-2" by B. G. 
Haskell et al 1997. This discussion of this book is incorporated herein by reference. This 
MPEG output stream is applied to a storage element 1 107 for storing program and 
summary segments. Program storage may be by memory circuits, disk recordings or any 
suitable means compatible with the content stored. 

Various stored matoial is selected under control of die software controller 1 103 
and tiansmitted to die digital to analog converter 1 109 to place the program or summary 
in a suitable form for the home television receiver. The analog signal is coupled to the 
home television receiver, via the Audio/Video RF CATV Modulator 1111, which 
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converts the signals to NTSC (National Television Standards Committee) standards for 



application to the home television receiver. 

The user/recipient exercises control of program summary selection by input to a 
user interface 1113 which commands are coupled to the software controller 1 103. 

S Commands of the software controller 1 103 are coupled to control the Modulator 1111, 
the storage element 1 107 and the Demodulator 1 101. A user Profile and Viewing history 
1 l is input is also connected the Software controller 1 103. Its input allows automatic 
selection of programming material to suit viewer desires and preferences. It may base 
these selections of stored demographic and prior selected subject matter. The software 

10 controller uses this information to self decide which sununaiies to store in order to 
effectively utilize the storage capacity of the storage medium 1 107. The user may 
transmit commands via the user interface 1 1 13 to view or delete stored programs and 
summaries in the storage medium 1 107. Control by the user may be exercised by using 
dialog boxes that are displayed on the screen of the receiver. Examples of dialog displays 

IS are shown in the FIGs. 7 and 8. Various types of dialog boxes are discussed below in the 
description of the controlling flow process of the software controller. The user may 
communicate with the interface by a remote controller. 

In system&jitilizing xDSL, for video delivery (FIG. 12), the STB is replaced by a 
Video Summary Server (VSS) 1203 included in the DSL networic. The programming is 

20 supplied to the user ftom the VSS 1203, via a POP 1205 to a modem 1207, via a xDSL 
Access Link 121 1, to the customer premises 1209. The VSS 1203 has capabilities and 
processes similar to that of the STB and can autonomously decide which summaries to 
receive and store based on a user profile and past usage history stored in the VSS. A 
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billing system 1213 is connected to the VSS via the POP 1205. The VSS may be 
integrated within the POP if desired or>e embodied as a separate unit within the network. 

An important aspect of the process is monitoring the capability of the network 
storage elements to store the needed summaries. The process of FIG. 14 is intended to 

S provide this capability* Starting at terminal block 1401 the controller and the storage 
medium it controls wait for input from the summary channels. The controller inquires, 
per decision block 1405 as to whether this simunary should be stored based on user 
selection, user preferences, etc. If the summary is not to be stored the flow process 
returns to block 1403 and waits for further input from the summary channel. 

10 If the summary is to be stored, the process inquires, as per decision block 1407, if 

there is sufficient storage to store the summary. With sufficient storage the smnmary is 
stored and/or appended to an existing summary, as per block 1409, with process flow 
returning to block 1403. 

If sufficient storage is not available, the summary is prioritized as per block 141 1 

15 as to whether it is desirable to the user and as per block 1413 is prioritized in relation to 
all other programs and summaries being processed. If the immediate sununary is highly 



rated, preexisting summaries are deleted, as per block 1415, to provide memory space for 
the new summary.. The flow process proceeds to block 1409 instructing storage of the 

■ 

new summary. 

20 One particular procedure of permitting interaction of a user with the system is 

presented as an illustrative embodiment, although many other processes may be used to 
enact the invention. A general understanding of this illustrative embodiment of user 
control may be obtained from the process disclosed in a flow chart disclosed in FIGs. 15, 
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16 and 17. The process begins at the start tenninal 2001 followed by user entry of a 
command as per block 2003. The software controller categorizes the command, as per 
block 2005, as a decision to change a chaxmel or not change a channel. If the command is 
to change a channel to another program channel the flow of the process returns to the 

5 start of the process since this is a routine change of channel If the command does not 
change the channel the process proceeds to a decision as specified by block 2007, which 
inquires if the subscriber subscribes to the service, requested If the user is not a 
subscriber to the service the flow process presents the user with a dialog box #1 such as 
mdicted by instruction box 2027 which presents a menu permitting the subscriber to enter 

10 a subscription to the service. The flow returns to the start of the flow process. 

If the user is a subscriber, the process proceeds to a decision shown in the 
decision block 2009 to determine if the user wishes to change to a summary channel S. If 
the user wishes to chaise to a summary channel, the user is presented with a screen 
menu, as per box 2008, similar to the box illustrated in the FIG. 7 presenting a listing of 

IS choices of readily available summaries for viewing. Subsequently the flow process 
returns to the start terminal. 

If the user does not wish to be presented with a menu of existing summaries the 
process proceeds tp decision block 201 1 which inquires if the user wishes to change to an 
interrupt channel I to create a simmiary while the user is absent from the television 

20 receiver. If such a summary is desired the process proceeds to set P equal to a program 
channel nxmiber as per block 2033 and indicator II, as per box 2034, is set to equal the 
index number associated with a current segment of the program being viewed. The 
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• 



interrupt- viewing screen (FIG. 8) is displayed as per block 2037 and the process returns 
to the start 

If no change to chaimel I is desired the next decision, as per block 2013» 
determines if a particular channel Id is requested on the summary channel S. If a 

s particular sununary id is requested the summary is presented, as per instruction block 
2038 and the process returns to the start. If no particular summary is requested, a 
decision, as per block 2015, determines if there is a command to return to the channel P 
on the charmel carrying the interrupted summary. If such a command has been issued the 
process returns the viewer to the program chaimel P, as per box 2016. The process 

10 returns to start to await a new conmiand 

If no return to the channel P is requested the viewer has a choice to view a 
summary of the missed content due to interruption as per the decision of block 201 7. If 
the missed content is to be viewed the index is set to II as per block 2045 and 12 is set to 
the current program segment, as per block 2047. An inquiry is performed, as per decision 

IS block 2049, as to whether the value I- 12 and if it is the process proceeds to instructions 
of block 2053 causing the program chaimel P to be transmitted to the viewer's television 
receiver and the flow returns to start. If I does not equal 12 the instructions of block 205 1 
set I - 1 + 1 and the instructions of block 2052 transmit the I-th summary segment to the 
viewer's television receiver. 

20 If the decision of block 2017 is not to view a summary of missed content a 

subsequent decision in block 2019 inquires if a link command has been issued (i.e., a link 
button, for example, has been actuated by the viewer). If not, flow returns to the start. If 
the link command has been actuated, an inquiry by block 2021 inquires if the stored 
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summary is being transmitted If stored summary is not being transmitted, flow proceeds 
to a subsequent inquiry of block 2023 to determine if a stored program is being 
transmitted. Again if a stored program is not being transmitted, an inquiry by block 2025 
inquires if a live program is being transmitted and if not flow returns to start. 

S If the decision of block 2021 determines that a stored summary is being 

transmitted, a subsequent instruction of block 2057 sets I equal to the index of the current 
summary segment. An inquiry is initiated to determine if the I-th program segment is 
stored in the STB for cable reception or in the VSS for xDSL reception. If the segment is 
stored, as determined by block 2059 the flow process proceeds to instruction block 2119. 

10 If the segment is not stored an inquiry of block 206 1 inquires if the program has been 
previously broadcast If it is not a dialog box #2 is transmitted to the viewer, as per box 



2062, which indicates to the viewer that the program has been previously broadcast but 
has not been recorded. If the program was not previously broadcast an inquiry of block 
2063 inquires if it is presently being broadcast If it is being broadcast the instructions of 

1 s block 207 1 switch the viewer to that program channel and the flow returns to start. If it 
is not being broadcast a query of block 2073 asks if the program is scheduled for future 
broadcast If it is so scheduled a dialog box #3 is presented to the viewer, as per block 
2075, so indicatiiig that the program is scheduled for future broadcast and makes an 
interactive offer to record it If the program is not scheduled for future broadcast another 

20 dialog box #4 is presented to the viewer, as per box 2076, indicating that the program is 
not recorded and is not scheduled for future broadcast. In both previous instances the 
flow returns to start. The program schedule 1002 in FIG. 10 is used to obtain the 



18 




Bhagavath; Markowitz; 0*Neil 
1999-0494 

infonnation about the schedule of a paiticular program. This information includes the 

unique ids that identify the program and its summaries. 

If the decision of block 2023 determines that the stored program is being 

transmitted, I is set equal to the index of the current program segment, as per box 2072, 
5 and subsequmt decision block 2077 presents the inquiry as to whether the I-th segment 

summary is stored in the STB or VSS. If it is not stored dialog box # 5 is transmitted to 

the viewer, as per box 2078, to indicate that a summary is not stored for this program. 
If the I-th summary segment is stored in the STB or VSS the I-th summary 

segment is sent to the viewer as instructed by block 2101. Decision block 2103 inquires 
10 if this summary uses two way links. If two-way links are not used the remaining 

summary segments are transmitted as per block 2111. As per subsequent instructions of 

block 2113 the entire summary is transmitted. The process subsequently retums to the 

start. 

If the two-way links are being used the (I+l)-th program segment is transmitted to 
15 the user as per block 2105 followed by transmission of the remaining program segment as 
per block 2107. Following this the entire program is transmitted, as per block 2019, and 
the process retums to start 

A determination by decision block 2025 that a live program is being transmitted 

mm* 

directs the flow process to block 2083 having the instruction to set I equal to the index of 
20 the current program segment An inquiry of block 2085 inquires if the I-th summary is 
stored in the STBA^SS. If it is not stored the system transmits the dialog message #6, as 
per block 2086, which indicates to the viewer that a summary is not stored for this 
program. If a summary is stored the I-th summary segment is transmitted to the viewer 
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as indicated in the block 2089. An inquiiy of block 2091 detennines if the summary uses 
two«way links. If two-way links are in use the user is switched to the program channel as 
per block 2095 and the process returns to start. 

If two-way links are not in use the remaining summary segments are transmitted 
S to the user as per block 2093. The entire summary is transmitted as per block 2097 and 
the process returns to start. 

If the decision block 2059 determines that an I-th program segment is stored in 
the STB/VSS the flow proceeds to block 21 19, as indicated above, whose instractions 
transmit the I-th program segment The inquiry of subsequent decision block 2121 
10 qumes if the summary uses two-way links. If not the remaining program segments are 
transmitted to the viewer as per block 2125 and the entire program is transmitted as per 
block 2127. If two-way links are in use the I-th summary segment is transmitted as per 



block 2123 and the remaining summary segments are also transmitted as per block 2 129 
resulting in transmission of the entire summary as per block 2131. The process returns to 



IS start. 



In the foregoing process the user is presented various dialog boxes on the 



television screen with which the user may interact or be provided with process-relevant 
information. These dialog boxes are listed to assist in imderstanding the above user 
process. Dialog Box #1 infonns the user that he/she has no subscription to the service. It 
20 provides interactive capability allowing the user to subscribe if desired. Dialog Box #2 
informs the user that the program desired has been previously broadcast but has not been 
recorded. Programs scheduled for future broadcast are announced by Dialog Box #3 as 
well as providing an interactive capability to give the user the capability to record it. 
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Dialog Box #4 infonns the user that a program previously broadcast and not recorded is 
not scheduled for future broadcast. The Dialog Box #S announces absence of sununaries 
stored for the present program. This list of Dialog Boxes is intended to be exemplary and 
is not intended to limit the scope of the process features of the invention. 

S The proceeding process illustrates one illustrative method of user interaction with 

the system. Many departures bom this scheme may be devised without departing from 
the spirit and scope of the invention. 

A unique feature of the process is the allowance of summary generation during 
actual live event programming. The summarization process is schematically illustrated in 

10 the FIG. 13 and involves the active monitoring of the program by an editor. As shown 
the Live program is recorded by camera and microphone and cast into a form (i.e., digital 
television signal) suitable for transmission by modulation and broadcast equipment. The 
program is transmitted by a program channel, via a POP, to an STB or VSS 1303 for 
application to a television receiver. 

15 The broadcast program (i.e., digital television signal) is also coupled to a 

computer (i.e., PC, workstation, etc.) 1305. The signal is buffered and displayed to allow 
an editor (i.e., live person), with appropriate editing software, to choose segments (i.e., 
program fragments) appropriate for creating a summary. These simunaries, as well as 
associated metadata, are transmitted to the STB or VSS. the metadata in accord with 

20 MPEG-2 standards is transmitted as part of an elementary data stream associated with the 
program. 
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This method allows the generation of summaries almost simultaneously with the 
program itself^ allowing a viewer to receive summaries only a few minutes subsequent to 
programming segments* 
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